Automated detection of lung slide to aid in diagnosis of pneumothorax

ABSTRACT

Methods and apparatuses for performing automated detection of lung slide using a computing device (e.g., an ultrasound system, etc.) are disclosed. In some embodiments, the techniques determine lung sliding using one or more neural networks. In some embodiments, the neural networks are part of a process that determines probabilities of the lung sliding at one or more M-lines. In some embodiments, the techniques display one or more probabilities of lung sliding in a B-mode ultrasound image.

PRIORITY

The present application claims the benefit of U.S. ProvisionalApplication No. 63/337,444, filed May 2, 2022, entitled “AUTOMATEDDETECTION OF LUNG SLIDE TO AID IN DIAGNOSIS OF PNEUMOTHORAX,” and isincorporated by reference in its entirety.

FIELD OF THE INVENTION

The embodiments disclosed herein relate generally to ultrasound imaging;more specifically, the embodiments disclosed herein relate to performingautomated detection of lung slide using ultrasound imaging systemsincluding the generation of visualizations (e.g., three-dimensionalimages) that indicate the presence of lung sliding.

BACKGROUND

Lung ultrasound (US) represents a novel and promising approach foraiding in the diagnosis of Pneumothorax (PTX), with high sensitivity andspecificity. More specifically, a determination of lung sliding ornon-sliding can aid in the diagnosis of PTX, and the diagnosis of PTXusing ultrasound equipment has been done and is determined using lungsliding/non-sliding metrics. The metrics usually involve motion withrespect to a pleural line in an ultrasound image. Currently, cliniciansevaluate the B-mode video clips for motion above and below the pleuralline. Additionally, clinicians use M-mode to look at the motion aboveand below the pleural line. These techniques have disadvantages in thatthey must be done by someone skilled in recognizing lung sliding and/orare time consuming and prone to user error. These disadvantages couldprevent the use of these techniques in real-time in certain situations,which could impact lifesaving efforts.

SUMMARY

Methods and apparatuses for performing automated detection of lungsliding using a computing device (e.g., an ultrasound system, etc.) aredisclosed. In some embodiments, the methods are implemented by acomputing device. In some embodiments, a method implemented by acomputing device for determining lung sliding includes receiving one ormore B-Mode ultrasound images that include a pleural line, generating afeature list from the one or more B-Mode ultrasound images, the featurelist indicating at least one feature of the pleural line, andgenerating, with a neural network implemented at least partially inhardware of the computing device and configured to process the featurelist and a B-Mode ultrasound image of the one or more B-Mode ultrasoundimages, a probability of the lung sliding.

In some other embodiments, a method implemented by a computing devicefor determining lung sliding includes generating B-Mode ultrasoundimages, determining an instruction for improving a quality of the B-Modeultrasound images, and displaying, on a user interface of the computingdevice, the instruction. The method also includes generating additionalB-Mode ultrasound images based on a user adjustment implemented based onthe instruction and generating, with a neural network implemented atleast partially in hardware of the computing device and based on one ormore of the additional B-Mode ultrasound images, a probability of thelung sliding.

In some other embodiments, an ultrasound system for determining lungsliding includes a memory to maintain ultrasound images and a medicalworksheet, a neural network implemented at least partially in hardwareof the ultrasound system to generate, based on one or more of theultrasound images, a probability of the lung sliding, and a processorsystem to populate, automatically and without user intervention inresponse to the neural network generating the probability, a field ofthe medical worksheet with an indicator of the lung sliding that isbased on the probability.

Other systems, machines and methods for automated detection of lungsliding are also described.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention will be understood more fully from the detaileddescription given below and from the accompanying drawings of variousembodiments of the invention, which, however, should not be taken tolimit the invention to the specific embodiments, but are for explanationand understanding only.

FIG. 1 illustrates some embodiments of an ultrasound machine.

FIGS. 2A and 2B illustrate examples of ultrasound images.

FIG. 3A illustrates an example of a good quality image.

FIG. 3B illustrates an example of a bad quality image.

FIG. 4 illustrates an example of a pleural line.

FIG. 5A illustrates M-mode images being constructed from M-line columnsof B-mode video frames.

FIG. 5B shows the processing of the M-mode images with a neural networkto generate probabilities of lung sliding at three M-lines.

FIG. 6 illustrates some embodiments of a system for performing lungsliding detection processing.

FIG. 7 illustrates a data flow diagram of some embodiments of a lungsliding detection process.

FIG. 8 illustrates a flow diagram of some embodiments of a process forgenerating M-mode ultrasound images from B-mode ultrasound images.

FIG. 9A illustrates a flow diagram of some embodiments of a process fordetermining lung sliding.

FIG. 9B illustrates some embodiments of a process for determining lungsliding in which additional probabilities of the lung sliding aregenerated and combined with other probabilities of lung sliding.

FIG. 10 illustrates a flow diagram of some embodiments of anotherprocess for determining lung sliding.

FIG. 11 illustrates a flow diagram of some other embodiments of aprocess for determining lung sliding

FIG. 12 illustrates a flow diagram of some other embodiments of aprocess for determining lung sliding.

FIG. 13 illustrates a flow diagram of some other embodiments of aprocess for determining lung sliding.

FIG. 14 illustrates an example of a user interface that may be displayedto an individual (e.g., clinician) using and/or viewing a display on anultrasound machine.

DETAILED DESCRIPTION

In the following description, numerous details are set forth to providea more thorough explanation of the present invention. It will beapparent, however, to one skilled in the art, that the present inventionmay be practiced without these specific details. In other instances,well-known structures and devices are shown in block diagram form,rather than in detail, in order to avoid obscuring the presentinvention.

Techniques are disclosed herein to automatically detect lung sliding inultrasound images generated with ultrasound systems. The detection oflung sliding may be used for aiding in the diagnosis of Pneumothorax(PTX). The automated detection of lung sliding on US can improvediagnostic accuracy and speed, as well as decrease patient managementtime.

In some embodiments, the ultrasound system automatically detects lungsliding or non-lung sliding in ultrasound images through the use of oneor more neural networks. These neural networks use trained models todetermine lung sliding to help reduce operator-to-operator variabilityand implement a consistent algorithm for detection of lung sliding. Insome embodiments, the neural networks aid the user by acquiring videoclips with acceptable quality to determine the presence of sliding inthe lung.

By automatically detecting lung sliding, the ability to diagnose PTX inreal-time with portable ultrasound equipment can have lifesavingimpacts, as the use of ultrasound equipment would enable the diagnosisof PTX at the point of care without needing to send patients or imagesto the radiology department. Furthermore, automated detection of lungsliding can improve diagnostic accuracy and speed, as well as decreasepatient management time.

Example automated detection algorithms and implementations are discussedin greater detail below.

FIG. 1 illustrates some embodiments of an ultrasound machine with thedisclosed technology. Referring to FIG. 1 , ultrasound transducer probe100 includes an enclosure 110 extending between a distal end portion 112and a proximal end portion 114. The ultrasound transducer probe 100 iselectrically coupled to an ultrasound imaging system 130 via a cable 118that is attached to the proximal end of the probe by a strain reliefelement 119. In some embodiments, ultrasound transducer probe 100 iselectrically coupled to an ultrasound imaging system 130 wirelessly.

A transducer assembly 120 having one or more transducer elements iselectrically coupled to the system electronics in ultrasound imagingsystem 130. In operation, transducer assembly 120 transmits ultrasoundenergy from the one or more transducer elements toward a subject andreceives ultrasound echoes from the subject. The ultrasound echoes areconverted into electrical signals by the one or more transducer elementsand electrically transmitted to the system electronics in ultrasoundimaging system 130 to form one or more ultrasound images.

Capturing ultrasound data from a subject using an exemplary transducerassembly (e.g., the transducer assembly 120) generally includesgenerating ultrasound, transmitting ultrasound into the subject, andreceiving ultrasound reflected by the subject. A wide range offrequencies of ultrasound may be used to capture ultrasound data, suchas, for example, low frequency ultrasound (e.g., less than 15 MHz)and/or high frequency ultrasound (e.g., greater than or equal to 15 MHz)can be used. Those of ordinary skill in the art can readily determinewhich frequency range to use based on factors such as, for example, butnot limited to, depth of imaging and/or desired resolution.

In some embodiments, ultrasound imaging system 130 includes ultrasoundsystem electronics 134 that comprises one or more processors, integratedcircuits, ASICs, FPGAs, and power sources to support the functioning ofultrasound imaging system 130 in a manner well-known in the art. In someembodiments, ultrasound imaging system 130 also includes ultrasoundcontrol subsystem 131 having one or more processors. At least oneprocessor, FPGA, or ASIC causes electrical signals to be sent to thetransducer(s) of probe 100 to emit sound waves and also receives theelectrical pulses from the probe that were created from the returningechoes. One or more processors, FPGAs, or ASICs process the raw dataassociated with the received electrical pulses and forms an image thatis sent to ultrasound imaging subsystem 132, which displays the image ondisplay screen 133. Thus, display screen 133 displays ultrasound imagesfrom the ultrasound data processed by the processor of ultrasoundcontrol subsystem 131.

In some embodiments, the ultrasound system can also have one or moreuser input devices (e.g., a keyboard, a cursor control device,microphone, camera, etc.) that inputs data and allows the taking ofmeasurements from the display of the ultrasound display subsystem, adisk storage device (e.g., hard, floppy, thumb drive, compact disks(CD), digital video discs (DVDs)) for storing the acquired images, and aprinter that prints the image from the displayed data. These devicesalso have not been shown in FIG. 1 to avoid obscuring the techniquesdisclosed herein.

In some embodiments, ultrasound system electronics 134 performsautomated detection of lung sliding. The automated detection of whetherlung sliding is present or not may aid clinicians in diagnosing orruling out Pneumothorax and includes benefits such as improveddiagnostic accuracy and speed, decreased patient management time,reduced operator-to-operator variability resulting from use of aconsistent algorithm for lung sliding.

In some embodiments, the automated detection of lung sliding isperformed using an automated artificial intelligence (AI) algorithm thatrelies on the observation of multiple frames to determine if sliding ispresent and its location within the body. In some embodiments, theautomated detection is performed by sending a series of images to aneural network (e.g., a convolutional neural network (CNN), SwinTransformer, etc.). The series of images may be ultrasound video clipsand may be sent as either a collection of stacked images into a singleCNN, a series of images into a RNN (Recurrent neural network), or atime-based AI model that is able to provide an indication (e.g., aprobability) of whether the images show that lung sliding is present.Given appropriate training data involving fully annotated images ofwhere sliding exists in each image, the model could learn to detectsliding and its location in the images. In some embodiments, as opposedto examining the frames as a whole, the automated detection processexamines single lines of the data. The single lines of data may beM-lines from M-mode images. These M-mode images may be generated in anumber of ways. For example, the M-mode images may be obtained throughM-mode acquisition where a collection of a single line of data isacquired at a fixed rate (for example 100 lines per second) for a periodof time (for example one second equals 100 data lines). Additionally oralternatively the M-mode images may be obtained by creating them fromB-mode images.

In some embodiments, the automated detection process detects lungsliding from this single M-mode strip (hereinafter “M-strip”) bycreating one or more M-mode images based on one or more M-lines. Thatis, an M-strip is a sequence of B-mode frames (e.g., 3 B-mode frames)out of which M-mode images are extracted at various M-lines. Details ofthese embodiments are described in more detail below. In someembodiments, the automated detection uses a neural network to examinethe single M-strip to determine if there is motion above and below thepleural line, thereby indicating that the lung has not collapsed. Insome embodiments, if the acquisition frame rate is high enough, theautomated detection process extracts multiple M-strips from a collectionof B-mode images (e.g., two-dimensional (2-D) video clips, etc.) anduses a neural network to detect lung sliding from the M-strips. In someembodiments, the automated detection process extracts M-mode lines at anangle to the vertical from each B-mode image in a technique oftenreferred to as anatomical M-mode and uses a neural network to examinethese lines to determine if lung sliding is present. In both thesecases, the neural network has a model that is trained using appropriatetraining data involving fully annotated images of where sliding existsin each image and learns to detect sliding and its location in inputimages.

FIGS. 2A and 2B illustrate examples of a B-mode image with a selectedhorizontal position (illustrated in the top middle portion of thefigure) and an M-strip at that location over a number of frames(illustrated below the B-mode image in the figure). In some embodiments,the M-strip is a 3-dimensional array of data (e.g., x and y dimensionsof the B-mode image along with the z dimension of time—i.e. frames). Insome embodiments, the M-strip is extracted from a sequence of B-modeimages and the M-mode image is reconstructed from 2-D ultrasound videoclips.

A lung with lung sliding (i.e., a lung which indicates a normal aerationpattern in an inflating and deflating lung) can be shown in an M-modepattern of uninterrupted horizontal lines superficial to the pleuralsurface with a granular pattern deep to this level. This is sometimesreferred to as a “seashore sign”. FIG. 2A illustrates a “seashore sign”in which there is a transition 203 between the “sea” and the “shore”where sliding is detected at pleural line 200 in M-mode image 202(generated from a number of frames of B-mode images 201) as indicated bythe motion above and below pleural line 200 of B-mode image 201. Incontrast, FIG. 2B illustrates pneumothorax (PTX) with a patternsometimes called the “stratosphere” or “bar code” sign 213 in an M-modeimage 212 (generated from a number of frames from B-mode images 211),indicating that there is no motion and thus no lung sliding at pleuralline 210 in B-mode image 210.

Using a neural network to automatically detect lung sliding by examiningultrasound images has a number of benefits, including, but not limitedto, its computational requirements are small and the data is easy toannotate (e.g., is sliding, or is not sliding).

One challenge with an automated detection process that uses M-mode linesis determining what lines to test. In some embodiments, thedetermination of which lines to test is done by first identifying aregion of interest (ROI) in an image where M-mode images should beextracted (e.g., suitable to extract M-lines) and tested. For instance,the ROI indicates the set of M-lines from which a selection made be madeto extract M-mode images. For example, any of the M-line locations(e.g., X image location) between the left and right portions of the ROI,and once the M-lines are selected, M-mode images are extracted from thatM-strip at those M-lines (e.g., X locations). In some embodiments, asdiscussed above, this ROI spans the pleural line in a rib space of thelung. In one example, more than one M-line from the region is tested toimprove the accuracy of the sliding determination. It is also likelythat different regions of the lung will have different levels of slidingdepending on the severity of the PTX observed.

Example Automatic Detection Embodiments

In some embodiments, the automated detection process has a number ofprocesses including determining image quality for lung slidingdetection, determining an ROI for lung sliding detection, determining anacceptable image quality for M-mode reconstruction regions, anddetermining lung sliding detection. Each of these operations isdescribed in more detail below.

Determination of Image Quality and Region of Interest (ROI) for LungSliding Detection

To ensure that the lung sliding detection is evaluated on acceptableimages, an AI model referred to herein as a neural network (e.g., a CNN,etc.) can be trained to recognize images that have acceptable qualityand an appropriate view for use in automated detection of lung sliding.In some embodiments, the determination of acceptable quality is based onone or more factors, such as but not limited to resolution, gain,brightness, clarity, centeredness, depth, recognition of the pleuralline, and recognition of a rib and/or rib shadow.

In some embodiments, the neural network recognizes appropriate views byrecognizing the images that have such expected features as a pleuralline and ribs in the image. For example, in some embodiments, the neuralnetwork recognizes a clear pleural line in the upper middle region of animage and seeing at least one rib shadow on one of the sides of theimage. In one embodiment, the neural network is trained to recognize thelocations of the pleural line via different methods. These methods caninclude, but are not limited to, the use of two points at the extents ofthe pleural line, left and right extents and center depth, asegmentation map, and a heat map.

In some embodiments, data output from the neural network, combined withheuristics, can be used to determine images that are acceptable, or goodquality, or determine that images are not acceptable, or bad quality.FIG. 3A illustrates an example of a good quality image. The neuralnetwork may determine an image is not acceptable, as having bad quality,because of one or more of its attributes such as, for example, imagesthat are too dark, too bright, too fuzzy, too deep, too shallow, notcentered, in addition to not recognizing expected features such as thepleural line and ribs; and the neural network may determine that animage is acceptable, e.g., as having good quality, when it does not haveany of these attributes that made the image unacceptable. FIG. 3Aillustrates an example of a good-quality image. FIG. 3B illustrates anexample of a poor-quality image. In some embodiments, the neural networkoutputs the good and bad quality indications as good/bad probability fora number of attributes.

In addition to computing a good/bad probability for a number ofattributes, the neural network can also detect the location of the twopoints (e.g., x,y locations or coordinates) that mark the left and rightedges, or end points, of the pleural line in the image. FIG. 4illustrates an example of a pleural line. Referring to FIG. 4 , markers401 and 402 indicate the end points of the pleural line.

In some embodiments, to determine the overall quality of a B-modeultrasound image, the good/bad probability generated by the neuralnetwork is used in combination with heuristic rules that use the x/ylocations of the pleural line. In some embodiments, the x locations areused to determine if the pleural line spans a prescribed distance withinthe image. In some embodiments, the prescribed distance is based on thepercentage of the image centered on the center of the image. Forexample, the line segment made by connecting the ROI points can crossthe center of the image. If the pleural line does not span theprescribed distance, then the image is considered bad. The y locationsof the pleural line can be used to determine if the image is too deep ortoo shallow. The location information can be used to determine theregion of interest (ROI) over which a metric for lung sliding iscomputed. For instance, the x locations of the pleural line from themodel can be used to determine a ROI that can be used to select M-linelocations for reconstructed M-mode images.

In some embodiments, the ultrasound system provides guidance or feedbackto the user in terms of identifying the aspects of the image that needto be adjusted to produce a better-quality image. For example, theultrasound system can indicate that the image is required to be centeredbetter, or the pleural line is too short or off screen. Based on thisguidance/feedback, the user is able to adjust the position of the probeto adjust the image accordingly. Examples of the guidance also includeto adjust depth up or down, adjust gain, move left or right (e.g., tocenter the window, etc.). Feedback information 403 in FIG. 4 is anexample of the feedback or guidance that may be provided on the image oranother portion of the display. In this case, feedback information 403guides the user to hold still the position as opposed to adjusting theposition of the probe. The feedback/guidance information may begenerated by a neural network. In some embodiments, without userintervention, the ultrasound system automatically turns on and collectsdata when a neural network indicates the view is good enough, andanalyzes sliding (by feeding images to the model).

Determination of Acceptable Quality for M-Mode Reconstruction Regions(M-Strips)

M-mode images can be reconstructed from an M-strip. Before constructingM-mode images from the M-strip, the frames can be examined to determineif the M-strip is acceptable for determining lung sliding. Thisdetermination can be based on the reported quality of each frame in theM-strip being of good quality. Additionally or alternatively, in someembodiments, the lung sliding detection process examines ROI points todetermine if there is too much motion. Excessive motion can make itdifficult to determine if there is lung sliding or not in thereconstructed M-mode. By looking for excessive motion, the M-strip ismarked as having good or bad quality. If the M-strip quality is bad,then it would not be used for lung sliding detection. In someembodiments, to detect motion within the M-strip frames, the change inthe x,y locations of the pleural line in consecutive B-mode frames canbe compared to each other to see if it exceeds a prescribed limit. Ifthe change in the x,y locations of the pleural line exceeds theprescribed limit, then the motion of the M-strip frame is too much foruse in determining if lung sliding exists or not. Note that thisdetermination of whether this is too much gross motion in the B-modeimages to use it for lung sliding detection can be made by a neuralnetwork. For example, the neural network can look at ROIs on multipleframes and if there is misalignment of the points throughout the frames,then the neural network would determine that the M-mode imagesreconstructed from the B-mode images would not be of good enoughquality.

Once an M-strip is designated as good quality, then M-mode images can bereconstructed for any of the M-lines in the B-mode image within the ROI.In some embodiments, the M-mode images may be reconstructed by takingthe vertical image pixels for a given M-line column from each frame(e.g., 25 frames) in the M-strip. This process can be repeated for allselected frames. Combining these vertical columns produces an M-modeimage with a pulse repetition frequency (PRF) equal to the frame rate ofthe video clip.

FIG. 5A illustrates M-mode images being constructed from M-line columnsof B-mode video frames. Referring to FIG. 5A, B-mode video framesforming M-strip 501 (e.g., 25 frames, etc.) are shown with M-linecolumns 502 highlighted. The same column of M-line columns 502 in eachof the B-mode video frames 501 can be combined to create the M-modeimages 503. While FIG. 5A only shows three M-mode images 503, there maybe less than or greater than three M-mode images 503 created from M-linecolumns 502 of B-mode video frames 501. Note that lung sliding can beperformed by evaluating multiple M-mode images constructed in thismanner. For example, a window that is three or more pixels wide may beexamined as a region of interest in the M-mode images 503. This windowmay be a sliding window that is examined to make a determination onwhether lung sliding does or does not exist somewhere in that region.

Alternatively or additionally to constructing M-mode images as describedabove, the lung sliding detection process can be run and lung slidingcan be detected on stored images (e.g., a CINE loop having a sequence ofdigital images from an ultrasound examination, etc.).

To reduce the motion in M-mode images, the ultrasound system can removemotion from the acquired B-mode frames used to construct the M-modeimages. This motion removal can be performed using algorithmictechniques such as, for example, but not limited to, warping ortranslation using optical flow or motion detection algorithms. Once themotion is removed from the sequence of B-mode images, then the M-modeimages can be constructed.

Determination of Lung Sliding

In some embodiments, an additional (e.g., a second) neural network istrained to discriminate between M-mode images that indicate lung slidingand M-mode images that indicate that there is no lung sliding. Thereconstructed M-mode images can be fed into this model to determine ifthere is sliding or not. In some embodiments, this determination is madebased on only one M-mode image. In some embodiments, this determinationis made based on multiple M-mode images. For example, depending on theavailable computing resources and response time, the ultrasound systemcan construct a variable number of M-mode images and pass them throughthe lung sliding model to determine if there is sliding or not. Thisdetection can be done for a number of M-mode images that are constructedfrom different M-line locations within the M-strip. This detection canalso be done for a number of M-strips (e.g., different sequences ofB-mode images that may or may not be contiguous in time). All of thelung sliding detection outputs can be combined in such a way as to get ahigher average accuracy than when looking at the lung sliding modeldetection from a single reconstructed M-mode. In some embodiments, thelung sliding detection outputs are combined using a mean function toachieve high accuracy.

FIG. 5B shows the processing of the M-mode images with a neural networkto generate probabilities of lung sliding at three M-lines. Referring toFIG. 5B, M-mode images 510 are input to neural network 520 to produceB-mode image 511 with M-lines 512 between pleural line end points 521and 522. The M-mode images 510 are examples of M-mode images generatedfrom an M-strip of B-mode images, such as M-mode images 503 in FIG. 5A,while M-lines 512 are examples of M-lines taken from a M-stripe such asM-line columns 502 of M-strip 501. While FIG. 5B shows three M-modeimages 510 being input to neural network 520, in alternativeembodiments, more or less than three M-mode images may be input intoneural network to detect lung sliding.

In some embodiments, M-lines 512 are displayed in the B-mode image 511with an indication indicating the probability of lung sliding or not.For example, one of M-lines 512 can be a particular gradient color(e.g., green) to indicate sliding, while another one of the M-lines 512can be displayed on the B-mode image 511 with a gradient colorindicating low or no probability of lung sliding (e.g., red). In thiscase illustrated in FIG. 5B, M-lines 512 that are displayed equal threein number. However, the techniques described here are not limited todisplaying only three M-lines. Note that there may be an M-line 512 forevery line in the M-mode images 510. In such a case, the lines couldindicate the start of lung sliding to a portion where there is no lungsliding. Additionally or alternatively, a user may select which M-linesare to be indicated in the B-mode image 511.

Alternative Refinements of the Lung Sliding Detection Auto-Refined LungSliding Detection of Suspected Cases

If non-sliding is detected on sparsely chosen M-lines in the region ofinterest, then the video clip could be further analyzed by running thelung sliding algorithm on a number of the M-lines (up to the number ofM-lines in the ROI). In other words, the analysis can be enhanced byanalyzing the video clip further by re-running the lung slidingalgorithm on a dense, rather than sparse, set of M-lines. These lungsliding detections could be averaged in the horizontal and/or temporaldirection, to filter out noise in the detection result. This resultcould be displayed graphically, e.g., a heat bar across the pleural lineto indicate the probability of lung sliding across the ROI. In someembodiments, to create the heat bar information, one set of M-modeimages would be reconstructed starting from a given frame (known as thestart frame). Then each of the reconstructed M-mode images can beprocessed through the lung sliding AI model to determine the probabilityof lung sliding at that M-line (e.g., the M-line corresponding to thereconstructed M-mode image). These probabilities could then be displayedas a heat bar where, for instance, solid red is 100% non-sliding andsolid green is 100% sliding. In some embodiments, all other probabilityvalues can be displayed as a gradient between solid red and solid green.Additionally or alternatively, the ultrasound system can display theprobabilities as a graph or impulse response with magnitudes betweenzero and one. To smooth out noise in the probabilities, theprobabilities could be filtered in the horizontal direction with asmoothing function.

To further enhance the fidelity of the probabilities used to generatethe heat bar, the above process could be repeated for two or more startframes and then the probabilities for each M-line from different startlines could be combined together to get a higher fidelity result. Thecombining algorithm can use a simple average or it could weight thehigher probability answers more than the lower probabilities.

Simultaneous B-Mode and M-Mode Image for Higher Fidelity Detection

Additional imaging states can be created that would enable a higherfidelity determination of lung sliding. Instead of reconstructing verylow-resolution M-mode images from the B-mode frames as described above,the ultrasound system can create an imaging state with interspersedadditional M-mode pings between the B-mode pings. There could be oneM-line chosen and pings could be transmitted and received for them-line. These pings could be acquired as fast as every other B-mode pingdown to as slow as one additional M-mode ping per frame. The trade-offhere would be the framerate of the B-mode video versus the resolution ofthe M-mode image. In some embodiments, the determination of which M-lineto fire is fixed, such as the center of the image or a percentage fromthe center of the image. Alternatively, in some embodiments, thedetermination of which M-line to fire is dynamically determined, suchas, for example, the center of the detected ROI or other locationswithin the ROI.

In some embodiments, multiple M-lines are selected and acquiredinterspersed with the B-mode images to allow the simultaneousacquisition of multiple higher resolution M-mode images. As with theacquisition of a single M-mode image interspersed with the B-modeframes, there would be a trade-off between the number of M-lines and thetemporal resolution of the M-mode images versus the frame rate of theB-mode images.

High Resolution Mapping Model

In one example, a CNN can be trained to map a lower PRF rate image intoa higher PRF rate image by training a super-resolution neural network toconstruct a higher resolution image. The ultrasound system can generateM-mode images of a first resolution from the M-strip as described above,and run these M-mode images through the super-resolution neural networkto create additional M-mode images having a higher resolution than thefirst resolution. These higher resolution M-mode images could be used asthe input for the lung sliding detection model to generate ahigh-accuracy probability of lung sliding. In some embodiments, theultrasound system generates an additional M-Mode ultrasound image basedon the M-Mode ultrasound image, where the additional M-Mode ultrasoundimage has a higher resolution than the M-Mode ultrasound image. In someof such embodiments, generating the probability of the lung sliding isbased on the additional M-Mode ultrasound image.

An Example of a Lung Sliding Detection System

FIG. 6 illustrates some embodiments of system for performing lungsliding detection processing. Referring to FIG. 6 , B-mode images from aB-mode image generator 601 are provided to quality check neural network(model) 602 and region of interest neural network (model) 603. In oneembodiment, the quality check neural network (model) 602 and the regionof interest neural network (model) 603 are separate neural networks. Insome embodiments, these neural networks are combined into one neuralnetwork. In still other embodiments, these networks share at least onecommon part and include other parts that are not shared between thesenetworks.

Quality check neural network 602 receives B-mode images from the B-modeimage generator 601 and determines whether each of B-mode images is ofsufficient quality to be used in the lung sliding detection process.Quality check neural network 602 determines the quality as describedabove and outputs quality level indications 610 for each of the B-modeimages. In some embodiments, the quality is output for display on adisplay screen (e.g., the display screen of an ultrasound machine, etc.)for the user to guide and improve their image acquisition.

Region of interest neural network 603 receives B-mode images from theB-mode image generator 601 and determines the location of the pleuralline 611. ROI neural network 603 outputs location information 611 foreach of the B-mode images. In some embodiments, the location informationincludes sets of coordinates of the end points of the pleural line. Insome embodiments, the coordinates are x, y coordinates of the end pointsof the pleural line in each of the B-mode images.

Quality level indication information 610 and location information 611are input to M-mode image generator 604 along with B-mode images fromthe B-mode image generator 601. In response to these inputs, M-modeimage generator 604 generates reconstructed M-mode images 612. In someembodiments, M-mode image generator 604 generates reconstructed M-modeimages 612 from B-mode images as described above. Additionally oralternatively, the M-mode images can be obtained through a well-knownM-mode image acquisition process.

Lung sliding detection neural network (model) 605 receives reconstructedM-mode images 612 and performs lung sliding detection on reconstructedM-mode images 612. In some embodiments, lung sliding detection isperformed as described above. As an output, lung sliding detectionneural network 605 generates lung sliding detection results 613. In someembodiments, the lung sliding detection results 613 includeprobabilities associated with each of the images for lung sliding. Thelung sliding detection results may be displayed on an ultrasound image,such as, for example, a B-mode image as described above. For example,the ultrasound system can display the lung sliding detection results aspart of a heat bar as previously described, and/or as part of a binaryicon that distinguishes lung sliding from no lung sliding, such as athumbs up/thumbs down indicator.

One or more of the neural networks of FIG. 6 can be implemented in anumber of different ways. In one embodiment, the neural networks includemodels that use an EfficientNet architecture, a convolutional neuralnetwork (CNN), and/or sequence models including recurrent neuralnetworks (RNN). Note that the detection techniques described herein canbe implemented with artificial intelligence (AI) or machine-learning(e.g., adaptive boosting (adaboost), deep-learning, supervised learningmodels, support vector machine (SVM), Gated Recurrent Unit (GRU),convolutional GRU (ConvGRU), long short-term memory (LSTM), etc., toprocess frame information in sequence, and the line), and/or anothersuitable detection method.

Example Flow Diagram of Lung Detection Processes

FIG. 7 illustrates a data flow diagram of some embodiments of a lungsliding detection process. The process can be performed by processinglogic that can include hardware (e.g., circuitry, dedicated logic,memory, etc.), software (such as is run on a general-purpose computersystem or a dedicated machine), firmware (e.g., software programmed intoa read-only memory), or combinations thereof. In some embodiments, theprocess is performed by one or more processors of a computing devicesuch as, for example, but not limited to, an ultrasound machine with anultrasound imaging subsystem.

Referring to FIG. 7 , the process begins by processing logic (e.g., oneor more memories) generating B-mode ultrasound images (processing block701). Processing logic generates one or more M-mode ultrasound imagescorresponding to one or more M-lines (processing block 702). In someembodiments, the one or more M-mode images are generated based on thepixels of the B-mode images and the one or more M-lines.

Processing logic generates one or more probabilities of lung sliding atone or more M-lines based on the one or more M-mode ultrasound images(processing block 703). In one embodiment, processing logic generatesthe one or more probabilities of lung sliding at the one or more M-linesusing a neural network. In some embodiments, the neural network isimplemented at least partially in hardware of a computing device.

After generating the one or more probabilities of lung sliding at one ormore M-lines, processing logic causes the display of representations of,or otherwise indicates, the probabilities of the lung sliding in atleast one B-mode ultrasound image (processing block 704).

FIG. 8 illustrates a flow diagram of some embodiments of a process forgenerating M-mode ultrasound images from B-mode ultrasound images. Theprocess can be performed by processing logic that can include hardware(e.g., circuitry, dedicated logic, memory, etc.), software (such as isrun on a general-purpose computer system or a dedicated machine),firmware (e.g., software programmed into a read-only memory), orcombinations thereof. In some embodiments, the process is performed byone or more processors of a computing device such as, for example, butnot limited to, an ultrasound machine with an ultrasound imagingsubsystem.

Referring to FIG. 8 , the process begins by processing logic generatinga quality level of the B-mode ultrasound images (processing block 801)and determining whether the quality level of the B-mode ultrasoundimages is above a quality threshold (processing block 802). In oneembodiment, processing logic generates the quality level of the B-modeultrasound images based on attribute quality probabilities and pairs ofcoordinates. Examples of attribute quality probabilities includeprobabilities for attribute qualities including a resolution, a gain, abrightness, a clarity, a centeredness, a depth, a recognition of thepleural line, a recognition of a rib, and the like. In some embodiments,processing logic generates the attribute quality probabilities for theB-mode images and pairs of coordinates that indicate edges (e.g., endpoints) of a pleural line in the B-mode images. In some embodiments,processing logic generates the attribute quality probabilities using aneural network. In some embodiments, the neural network is implementedat least partially of hardware of a computing device.

After determining whether the quality level is above a qualitythreshold, processing logic generates one or more M-mode ultrasoundimages (processing block 803). In some embodiments, processing logicgenerates one or more M-mode ultrasound images in response to thequality level being above the quality threshold. In other words, theM-mode ultrasound images are only generated if the quality of the B-modeimages is above the quality threshold.

In some embodiments, processing logic generates additional M-modeultrasound images by transmitting M-mode ultrasound signals that areinterspersed with the B-mode ultrasound signals used to generate theB-mode ultrasound images.

Thereafter, processing logic then generates one or more probabilities oflung sliding at one or more M-lines of the M-mode ultrasound images(processing block 804). In some embodiments, the one or moreprobabilities are based on the M-mode ultrasound images generated fromthe B-mode ultrasound images. In some embodiments, the one or moreprobabilities are also based on the additional M-mode ultrasound imagesgenerated by transmitting M-mode ultrasound signals that areinterspersed with the B-mode ultrasound signals used to generate theB-mode ultrasound images. In one embodiment, the one or moreprobabilities based on the additional M-mode ultrasound are generatedusing a neural network.

FIG. 9A illustrates a flow diagram of some embodiments of a process fordetermining lung sliding. The process can be performed by processinglogic that can include hardware (e.g., circuitry, dedicated logic,memory, etc.), software (such as is run on a general-purpose computersystem or a dedicated machine), firmware (e.g., software programmed intoa read-only memory), or combinations thereof. In some embodiments, theprocess is performed by one or more processors of a computing devicesuch as, for example, but not limited to, an ultrasound machine with anultrasound imaging subsystem.

Referring to FIG. 9A, the process begins by processing logic generatingattribute quality probabilities for B-mode ultrasound images and pairsof coordinates that indicate edges of a pleural line in the B-modeultrasound images (processing block 901). In some embodiments, theseattribute quality probabilities for B-mode ultrasound images and pairsof coordinates are generated with a first neural network implemented atleast partially in hardware of a computing device.

Next, processing logic determines a region of interest in the B-modeultrasound images (processing block 902). In some embodiments, theregion of interest in the B-mode ultrasound images is determined basedon the previously-generated pairs of coordinates.

Processing logic also determines a quality level of the B-modeultrasound images as acceptable for determining the lung sliding(processing block 903). In some embodiments, the determination that theB-mode ultrasound images have a quality level that is acceptable fordetermining the lung sliding is determined based on thepreviously-generated attribute quality probabilities and an amount ofmotion in the region of interest. In some embodiments, the attributequality probabilities indicate a probability of at least one attributequality taken from the group consisting of resolution, gain, brightness,clarity, centeredness, depth, recognition of the pleural line, andrecognition of a rib.

In some embodiments, determining the quality level as acceptableincludes, for each of the B-mode ultrasound images, determining ahorizontal span of the pleural line and comparing the horizontal span toa threshold distance. In some embodiments, determining the horizontalspan of the pleural line is performed based on horizontal components ofthe pairs of coordinates. In some embodiments, the process includesprocessing logic setting the threshold distance to be a percentage of asize of at least one of the B-mode ultrasound images. For example, insome embodiments, to be considered good quality, the pleural line mustbe located between 20% and 60% of the image vertically and the pleuralline should cross the middle of the image. In some embodiments,determining the quality level as acceptable includes, for each of theB-mode ultrasound images, determining a depth of said each of the B-modeultrasound images based on vertical components of the pairs ofcoordinates.

Using the B-mode ultrasound images, processing logic generates one ormore M-mode ultrasound images corresponding to one or more M-lines inthe region of interest (processing block 904). In some embodiments, theM-mode ultrasound images are from columns of pixels in each of theB-mode ultrasound images that correspond to the one or more M-lines.

Based on the one or more M-mode ultrasound images, processing logicgenerates probabilities of the lung sliding at the one or more M-lines(processing block 905). In some embodiments, processing logic generatesprobabilities of the lung sliding at the one or more M-lines with aneural network. The neural network may be implemented at least partiallyin the hardware of the computing device (e.g., an ultrasound machine,such as the ultrasound system 130 in FIG. 1 ).

Processing logic can also display visual representations of the one ormore M-lines that indicate the probabilities of the lung sliding at theone or more M-lines (processing block 906). Color-coded versions of theM-lines 512 drawn in FIG. 5B are examples of the visual representationsof the one or more M-lines that indicate the probabilities with colors.In some embodiments, processing logic displays the representations ofthese M-lines in a B-mode ultrasound image. In some embodiments,processing logic displays the visual representation horizontally acrossthe region of interest, such as via a heat bar, as previously described.In some embodiments, the process of generating the visual representationincludes processing logic filtering the probabilities. The probabilitiescan be filtered with a smoothing function in a horizontal direction.

In some embodiments, the one or more M-mode ultrasound images includemultiple M-mode ultrasound images and the one or more M-lines includemultiple M-lines across the region of interest. In some embodiments, insuch a case, the process generates a visual representation of theprobabilities of the lung sliding at the multiple M-lines and displaysthe visual representation horizontally across the region of interest.

In such a case, processing logic can generate the multiple M-modeultrasound images based on a first start frame of the B-mode ultrasoundimages. In some embodiments, the process includes generating additionalM-mode ultrasound images based on a second start frame of the B-modeultrasound images and generating additional probabilities of the lungsliding at the multiple M-lines. The process can also include combiningthe probabilities and the additional probabilities to form combinedprobabilities of the lung sliding at the multiple M-lines. After formingthe combined probabilities, the process generates and displays a visualrepresentation of the combined probabilities. In some embodiments,processing logic generates the additional probabilities of the lungsliding at the multiple M-lines with a neural network and based on theadditional M-mode ultrasound images.

FIG. 9B illustrates some embodiments of a process for determining lungsliding in which the additional probabilities of the lung sliding aregenerated and combined with other probabilities of lung sliding. Theprocess can be performed by processing logic that can include hardware(e.g., circuitry, dedicated logic, memory, etc.), software (such as isrun on a general-purpose computer system or a dedicated machine),firmware (e.g., software programmed into a read-only memory), orcombinations thereof. In some embodiments, the process is performed byone or more processors of a computing device such as, for example, butnot limited to, an ultrasound machine with an ultrasound imagingsubsystem.

Referring to FIG. 9B, the process begins by processing logic generatingattribute quality probabilities for B-mode ultrasound images and pairsof coordinates that indicate edges of a pleural line in the B-modeultrasound images (processing block 911). In some embodiments, theseattribute quality probabilities for B-mode ultrasound images and pairsof coordinates are generated with a first neural network implemented atleast partially in hardware of a computing device.

Next, processing logic determines a region of interest in the B-modeultrasound images (processing block 912). In some embodiments, theregion of interest in the B-mode ultrasound images is determined basedon the previously-generated pairs of coordinates.

Processing logic also determines a quality level of the B-modeultrasound images as acceptable for determining the lung sliding(processing block 913). In some embodiments, the determination that theB-mode ultrasound images have a quality level that is acceptable fordetermining the lung sliding is determined based on thepreviously-generated attribute quality probabilities and an amount ofmotion in the region of interest. In some embodiments, the attributequality probabilities indicate a probability of at least one attributequality taken from the group consisting of resolution, gain, brightness,clarity, centeredness, depth, recognition of the pleural line, andrecognition of a rib.

In some embodiments, determining the quality level as acceptableincludes, for each of the B-mode ultrasound images, determining ahorizontal span of the pleural line and comparing the horizontal span toa threshold distance. In some embodiments, determining the horizontalspan of the pleural line is performed based on horizontal components ofthe pairs of coordinates. In some embodiments, the process includesprocessing logic setting the threshold distance to be a percentage of asize of at least one of the B-mode ultrasound images. For example, insome embodiments, to be considered good quality, the pleural line mustbe located between 20% and 60% of the image vertically and the pleuralline should cross the middle of the image. In some embodiments,determining the quality level as acceptable includes, for each of theB-mode ultrasound images, and determining a depth of each of theseB-mode ultrasound images based on vertical components of the pairs ofcoordinates.

Using the B-mode ultrasound images, processing logic generates one ormore M-mode ultrasound images corresponding to one or more M-lines inthe region of interest (processing block 914). In some embodiments, theM-mode ultrasound images are from columns of pixels in each of theB-mode ultrasound images that correspond to the one or more M-lines.

Based on the one or more M-mode ultrasound images, processing logicgenerates probabilities of the lung sliding based on one or more M-modeimages (e.g., at the one or more M-lines) (processing block 915). Insome embodiments, processing logic generates probabilities of the lungsliding at the one or more M-lines with a neural network. The neuralnetwork may be implemented at least partially in the hardware of thecomputing device (e.g., an ultrasound machine, such as the ultrasoundsystem 130 in FIG. 1 ).

Processing logic then generates additional M-mode ultrasound imagesbased on a second start frame of the B-mode ultrasound images(processing block 916) and additional probabilities of the lung slidingbased on the additional M-mode ultrasound images (processing block 917).In some embodiments, these are generated in the same manner as describedabove in conjunction with processing blocks 914 and 915.

Processing logic combines the multiple probabilities generated fromprocessing block 915 with the additional probabilities to form combinedprobabilities of the lung sliding (processing block 916).

Processing logic can also generate a visual representation of thecombined probabilities (processing block 919) and display the visualrepresentation (processing block 920). Color-coded versions of theM-lines 512 drawn in FIG. 5B are examples of the visual representationsof the one or more M-lines that indicate the probabilities with colors.In some embodiments, processing logic displays the representations ofthese M-lines in a B-mode ultrasound image. In some embodiments,processing logic displays the visual representation horizontally acrossthe region of interest, such as via a heat bar, as previously described.In some embodiments, the process of generating the visual representationincludes processing logic filtering the probabilities. The probabilitiescan be filtered with a smoothing function in a horizontal direction.

FIG. 10 illustrates a flow diagram of some embodiments of anotherprocess for determining lung sliding. The process can be performed byprocessing logic that can include hardware (e.g., circuitry, dedicatedlogic, memory, etc.), software (such as is run on a general-purposecomputer system or a dedicated machine), firmware (e.g., softwareprogrammed into a read-only memory), or combinations thereof. In someembodiments, the process is performed by one or more processors of acomputing device such as, for example, but not limited to, an ultrasoundmachine with an ultrasound imaging system.

Referring to FIG. 10 , the process begins by processing logic generatingB-mode ultrasound images (processing block 1001). In some embodiments,the B-mode ultrasound images are generated in a manner well-known in theart.

In some embodiments, processing logic determines the quality levels ofthe B-mode ultrasound images (processing block 1002). In someembodiments, processing logic determines the quality levels using aprocess that includes generating pairs of coordinates that indicateedges of a pleural line in the B-mode ultrasound images, determining aregion of interest in the B-mode ultrasound images based on the pairs ofcoordinates, and determining an amount of motion in the region ofinterest. In some embodiments, the pairs of coordinates that indicateedges of a pleural line in the B-mode ultrasound images are generatedwith a neural network. The neural network may be in addition to theneural network that generates probabilities of lung sliding at anM-line. In some embodiments, the neural network that generates the pairsof coordinates is implemented at least partially in the hardware of theultrasound system.

In some embodiments, processing logic determines the quality levelsusing a process that includes generating, with an additional neuralnetwork implemented at least partially in the hardware of the ultrasoundsystem, pairs of coordinates that indicate edges of a pleural line inthe B-mode ultrasound images. The process for determining the qualitylevels can also include determining a horizontal span of the pleuralline based on the pairs of coordinates, and comparing the horizontalspan to a threshold distance. In some embodiments, the processing logicgenerates pairs of coordinates that indicate edges of a pleural line inthe B-mode ultrasound images using a neural network. The neural networkcan be in addition to the neural network that generates probabilities oflung sliding at an M-line. In some embodiments, the neural network thatgenerates the pairs of coordinates is implemented at least partially inthe hardware of the ultrasound system.

In some embodiments, processing logic determines the quality levelsusing a process that includes generating attribute quality probabilitiesfor the B-mode ultrasound images that indicate a probability of at leastone attribute quality taken from the group consisting of a resolution, again, a brightness, a clarity, a centeredness, a depth, a recognition ofa pleural line, and a recognition of a rib. In some embodiments,processing logic generates attribute quality probabilities for theB-mode ultrasound images using a neural network. The neural network maybe in addition to the neural network that generates probabilities oflung sliding at an M-line. In some embodiments, the neural network thatgenerates the attribute quality probabilities is implemented at leastpartially in the hardware of the ultrasound system.

Then processing logic discards a first portion of the B-mode ultrasoundimages based on the quality levels of the B-mode ultrasound images(processing block 1003) while maintaining a second portion of the B-modeultrasound images based on their quality levels (processing block 1004).In some embodiments, the probability of the lung sliding is based on theretained portion of the B-mode ultrasound images. Note also that in someembodiments, the quality may also be displayed to the user.

Using the retained B-mode ultrasound images, processing logic generatesan M-mode ultrasound image corresponding to an M-line (processing block1005). Note that this process may be repeated such that multiple M-modeultrasound images are generated. In some embodiments, processing logicgenerates the M-mode images based on pixels in the B-mode ultrasoundimages that correspond to the M-line.

In some embodiment, processing logic also generates an additional M-modeultrasound image based on the M-mode ultrasound image. The additionalgenerated M-mode ultrasound image has a higher resolution than theconstructed low-resolution M-mode ultrasound image, and the probabilityof the lung sliding is based on the additional M-mode ultrasound image.In some embodiments, generating the additional M-mode ultrasound imageis performed with a neural network, such as a super-resolution neuralnetwork. The neural network (e.g., super-resolution neural network) canbe in addition to the neural network that generates probabilities oflung sliding at an m-line. In some embodiments, the neural network(e.g., super-resolution neural network) is implemented at leastpartially in the hardware of the ultrasound system.

Based on the M-mode ultrasound image, processing logic generates aprobability of the lung sliding at each M-line (processing block 1006).In some embodiments, processing logic generates a probability of thelung sliding at the M-line using a neural network. In some embodiments,the neural network is implemented at least partially in the hardware ofthe ultrasound system.

After generating a probability of the lung sliding at the M-line,processing logic generates an additional B-mode ultrasound image(processing block 1007) and indicates in the additional B-modeultrasound image the probability of the lung sliding (processing block1008).

In some embodiments, the lung sliding detection generates lung slidingprobabilities based on B-mode. In some embodiments, the ultrasoundsystem uses a neural network implemented at least partially in hardwareof the ultrasound system and based on B-mode ultrasound images. In someembodiments, the neural network generates a probability of lung slidingby operating on clips of B-mode images. For example, the neural networkmay be fed features out of a model (e.g., a QC (QCRY) model) andfeatures of B-mode image are generated from layers of QC model. Thesefeatures can be used as conditional/additional/secondary input, withB-mode images as main input. In some embodiments, the system selectsB-mode images to reduce the burden on the neural network. For example,the system can discard redundant images which would make neural networkrun more slowly. In some embodiments, the neural network is only fed aregion of interest (ROI) of each B-mode image, instead of the fullimage. The ROI could be based on certain provided width and/or height.The ROI can be selected so as to capture the pleural line. If specifyingthe width and/or height of the ROI of interest, the location specifiedmay indicate a distance from an end point of the pleural line to an edgeof the image in both horizontal and vertical directions (left and/orright of the pleural line and above and/or below the pleural line). Thisresults in a ROI in which the pleural line is centered. Additionally oralternatively, a height and width of the ROI can be based on thedistance specified by the neural network or by the ultrasound machine.

FIG. 11 illustrates a flow diagram of some other embodiments of aprocess for determining lung sliding. The process can be performed byprocessing logic that can include hardware (e.g., circuitry, dedicatedlogic, memory, etc.), software (such as is run on a general-purposecomputer system or a dedicated machine), firmware (e.g., softwareprogrammed into a read-only memory), or combinations thereof. In someembodiments, the process is performed by one or more processors of acomputing device such as, for example, but not limited to, an ultrasoundmachine with an ultrasound imaging system.

Referring to FIG. 11 , the process begins by processing logic receivingone or more B-Mode ultrasound images that include a pleural line(processing block 1101) and generating a feature list from the one ormore B-Mode ultrasound images, where the feature list indicates at leastone feature of the pleural line (processing block 1102).

After generating the feature list, processing logic generates aprobability of the lung sliding with a neural network implemented atleast partially in hardware of the computing device and configured toprocess the feature list and a B-Mode ultrasound image of the one ormore B-Mode ultrasound images (processing block 1103). In someembodiments, generating the probability of the lung sliding includesactivating the neural network to process the feature list and the B-Modeultrasound image automatically and without user intervention based onthe B-Mode ultrasound image having a quality level above a thresholdquality level.

In some embodiments, the process set forth in FIG. 11 further comprisesdetermining a region of interest in the B-Mode ultrasound image based ona location of the pleural line in the B-Mode ultrasound image. In someof such embodiments, generating the probability of the lung sliding isbased on pixels of the B-Mode ultrasound image that are included in theregion of interest and not based on additional pixels of the B-Modeultrasound image that are not included in the region of interest. Insome embodiments, the location indicates a distance from an end point ofthe pleural line to an edge of the B-Mode ultrasound image, and thedetermining the region of interest is based on the distance.

In some embodiments, the process set forth in FIG. 11 further comprisesgenerating an additional probability of the lung sliding based on anadditional B-Mode ultrasound image of the one or more B-Mode ultrasoundimages. In some of such embodiments, generating the additionalprobability of the lung sliding is based on the feature list.

In some embodiments, the process set forth in FIG. 11 further comprisesgenerating an additional probability of the lung sliding based on anadditional B-Mode ultrasound image of the one or more B-Mode ultrasoundimages and generating an additional feature list from the additionalB-Mode ultrasound image, wherein the generating the additionalprobability of the lung sliding is based on the additional feature list.

In some embodiments, the process set forth in FIG. 11 further comprisesgenerating an additional probability of the lung sliding based on anadditional B-Mode ultrasound image of the one or more B-Mode ultrasoundimages, merging the probability and the additional probability to form amerged probability of the lung sliding, and displaying, in a userinterface of the computing device, a representation of the mergedprobability of the lung sliding.

In some embodiments, the process set forth in FIG. 11 further comprisesdetermining an additional B-Mode ultrasound image of the one or moreB-Mode ultrasound images as redundant to the B-Mode ultrasound image,and discarding the additional B-Mode ultrasound image from the one ormore B-Mode ultrasound images to prevent the neural network fromprocessing the additional B-Mode ultrasound image.

FIG. 12 illustrates a flow diagram of some other embodiments of aprocess for determining lung sliding. The process can be performed byprocessing logic that can include hardware (e.g., circuitry, dedicatedlogic, memory, etc.), software (such as is run on a general-purposecomputer system or a dedicated machine), firmware (e.g., softwareprogrammed into a read-only memory), or combinations thereof. In someembodiments, the process is performed by one or more processors of acomputing device such as, for example, but not limited to, an ultrasoundmachine with an ultrasound imaging system.

Referring to FIG. 12 , the process begins by processing logic generatingB-Mode ultrasound images (processing block 1201). In some embodiments,the B-Mode ultrasound images include a pleural line and the quality ofthe B-Mode ultrasound images is based on a location of the pleural linein the B-Mode ultrasound images.

After generating the B-Mode ultrasound images, processing logicdetermines an instruction for improving a quality of the B-Modeultrasound images (processing block 1202) and displays, on a userinterface of the computing device, the instruction (processing block1203). In some embodiments, the instruction includes at least one ofguidance to move an ultrasound probe, an adjustment of an imagingparameter, and a recommendation for selecting the neural network from alist of neural networks available on the computing device.

Processing logic generates one or more additional B-Mode ultrasoundimages based on a user adjustment implemented based on the instruction(processing block 1204). In some embodiments, when generating multipleB-Mode ultrasound images, the process also includes determiningredundant B-Mode ultrasound images of the multiple B-Mode ultrasoundimages; and excluding one or more of the redundant B-Mode ultrasoundimages from the generating the probability of the lung sliding bypreventing the neural network from processing data determined from theone or more of the redundant B-Mode ultrasound images.

After generating one or more additional B-Mode ultrasound images basedon a user adjustment implemented based on the instruction, processinglogic generates a probability of the lung sliding, based on one or moreof the additional B-Mode ultrasound images, with a neural network(processing block 1205). In some embodiments, the neural network isimplemented at least partially in hardware of the computing device. Insome embodiments, generating the probability of the lung slidingincludes activating the neural network automatically and without userintervention based on the one or more of the additional B-Modeultrasound images having a quality level above a threshold qualitylevel.

In some embodiments, the process set forth in FIG. 12 further comprisesdetermining a region of interest in the one or more of the additionalB-Mode ultrasound images. In some of such embodiments, generating theprobability of the lung sliding is based on pixels of the one or more ofthe additional B-Mode ultrasound images that are included in the regionof interest and not based on additional pixels of the one or more of theadditional B-Mode ultrasound images that are not included in the regionof interest. In some of such embodiments, the determining the region ofinterest is based on a pleural line in the one or more of the additionalB-Mode ultrasound images.

In some embodiments, a computing system (e.g., the ultrasound system)that performs the lung sliding detection described herein includes anenhanced workflow and user interface. In this case, the computing systempre-populates a worksheet with data from the neural network or AI basedmachine. The worksheet is then displayed, or otherwise provided, to aclinician for verification (e.g., send a “confirm based on ownassessment” message). In one embodiment, the ultrasound system includesan ultrasound probe with an inertial measurement unit (IMU) and detectswhere in the protocol the scan is, and then performs the auto-populationdescribed above. The clinician usually follows a specific patternaccording to the protocol, so with IMU data, the ultrasound system candetermine that they have moved from L1 to L2. In some embodiments, theuser interface can indicate the detection of a lung pulse.

FIG. 13 illustrates a flow diagram of some other embodiments of aprocess for determining lung sliding using an enhanced workflowdescribed above. The process can be performed by processing logic thatcan include hardware (e.g., circuitry, dedicated logic, memory, etc.),software (such as is run on a general-purpose computer system or adedicated machine), firmware (e.g., software programmed into a read-onlymemory), or combinations thereof. In some embodiments, the process isperformed by one or more processors of a computing device such as, forexample, but not limited to, an ultrasound machine with an ultrasoundimaging system.

Referring to FIG. 13 , the process begins by processing logicmaintaining, in a memory of an ultrasound system, ultrasound images anda medical worksheet (processing block 1301) and generating a probabilityof the lung sliding based on one or more of the ultrasound images(processing block 1302). In some embodiments, the processing logicgenerates the probability of lung sliding using a neural networkimplemented at least partially in hardware of the ultrasound system.

After generating the probability of lung sliding, processing logicpopulates, automatically and without user intervention, a field of themedical worksheet with an indicator of the lung sliding that is based onthe probability (processing block 1303). In some embodiments, theprocessing logic populates the field of the medical worksheet with anindicator of the lung sliding in response to the neural networkgenerating the probability.

In some embodiments, the process set forth in FIG. 13 further comprisesdisplaying, in a display screen (e.g., an ultrasound system displayscreen), the medical worksheet including the field populated with theindicator of the lung sliding and a request for a user confirmation ofthe indicator of the lung sliding (processing block 1304).

In some embodiments, the process set forth in FIG. 13 further comprisesgenerating position data, wherein determining the field of the medicalworksheet is based on the position data. In some of such embodiments,the position data is generated by a position sensor of an ultrasoundprobe of the ultrasound system.

In some embodiments, the process set forth in FIG. 13 further comprisesscanning lung regions with an ultrasound probe of the ultrasound systembased on position data generated by a position sensor of the ultrasoundprobe. These lung regions can be indicated in the medical worksheet. Insome embodiments, the ultrasound system also includes an ultrasoundprobe having a position sensor configured to generate position data. Insuch a case, the processor system can be implemented to indicate in themedical worksheet, based on the position data, lung regions scanned withthe ultrasound probe.

FIG. 14 illustrates an example of a user interface that may be displayedto an individual (e.g., clinician) using and/or viewing a display on anultrasound machine.

The systems, devices, and methods disclosed herein constitute numerousadvantages over conventional ultrasound systems, devices and methodsthat do not implement automated detection of lung slide to aid in thediagnosis of PTX. For instance, the ultrasound systems disclosed hereincan reliably diagnose PTX in real-time with portable ultrasoundequipment, which simply cannot be done with conventional ultrasoundsystems due to the time required to operate the conventional ultrasoundsystems and the errors introduced by the operator. Consequently, theultrasound systems can more accurately and more quickly diagnose PTXthan conventional ultrasound systems and have lifesaving impacts at thepoint of care.

Moreover, by using the ultrasound systems disclosed herein, the burdenof resources of a care facility is reduced compared to the use ofconventional ultrasound systems. This advantage is because the use ofthe ultrasound systems disclosed herein can result in successfuldiagnosis of PTX with the ultrasound system alone, without the need tosend patients to another imaging department, such as a radiologydepartment. In contrast, because conventional ultrasound systems may notsuitably diagnosis PTX, as described above, they may require the use ofadditional imaging, and therefore place higher burdens on the resourcesof the care facility than the ultrasound systems disclosed herein.Hence, the ultrasound systems disclosed herein can make the carefacility operate more efficiently and thus provide better patient care,compared to conventional ultrasound systems.

Further, because the ultrasound systems disclosed herein operate morequickly than conventional ultrasound systems that do not implementautomated detection of lung slide to aid in the diagnosis of PTX, theoperator can perform a more comprehensive ultrasound examination in agiven amount of time using the ultrasound systems disclosed hereincompared to conventional ultrasound systems. Accordingly, the patientmay receive better care with the ultrasound systems disclosed hereincompared to conventional ultrasound systems.

There are a number of example embodiments described herein.

Example 1 is method implemented by a computing device for determininglung sliding, where the method comprises: receiving one or more B-Modeultrasound images that include a pleural line; generating a feature listfrom the one or more B-Mode ultrasound images, the feature listindicating at least one feature of the pleural line; and generating,with a neural network implemented at least partially in hardware of thecomputing device and configured to process the feature list and a B-Modeultrasound image of the one or more B-Mode ultrasound images, aprobability of the lung sliding.

Example 2 is the method of example 1 that may optionally include thatthe generating the probability of the lung sliding includes activatingthe neural network to process the feature list and the B-Mode ultrasoundimage automatically and without user intervention based on the B-Modeultrasound image having a quality level above a threshold quality level.

Example 3 is the method of example 1 that may optionally includedetermining a region of interest in the B-Mode ultrasound image based ona location of the pleural line in the B-Mode ultrasound image, whereinthe generating the probability of the lung sliding is based on pixels ofthe B-Mode ultrasound image that are included in the region of interestand not based on additional pixels of the B-Mode ultrasound image thatare not included in the region of interest.

Example 4 is the method of example 3 that may optionally include thatthe location indicates a distance from an end point of the pleural lineto an edge of the B-Mode ultrasound image, and the determining theregion of interest is based on the distance.

Example 5 is the method of example 1 that may optionally includegenerating an additional probability of the lung sliding based on anadditional B-Mode ultrasound image of the one or more B-Mode ultrasoundimages.

Example 6 is the method of example 5 that may optionally include thatthe generating the additional probability of the lung sliding is basedon the feature list.

Example 7 is the method of example 5 that may optionally includegenerating an additional feature list from the additional B-Modeultrasound image, wherein the generating the additional probability ofthe lung sliding is based on the additional feature list.

Example 8 is the method of example 5 that may optionally include mergingthe probability and the additional probability to form a mergedprobability of the lung sliding, and displaying, in a user interface ofthe computing device, a representation of the merged probability of thelung sliding.

Example 9 is the method of example 1 that may optionally includedetermining an additional B-Mode ultrasound image of the one or moreB-Mode ultrasound images as redundant to the B-Mode ultrasound image;and discarding the additional B-Mode ultrasound image from the one ormore B-Mode ultrasound images to prevent the neural network fromprocessing the additional B-Mode ultrasound image.

Example 10 is a method implemented by a computing device for determininglung sliding, where the method comprises: generating B-Mode ultrasoundimages; determining an instruction for improving a quality of the B-Modeultrasound images; displaying, on a user interface of the computingdevice, the instruction; generating additional B-Mode ultrasound imagesbased on a user adjustment implemented based on the instruction; andgenerating, with a neural network implemented at least partially inhardware of the computing device and based on one or more of theadditional B-Mode ultrasound images, a probability of the lung sliding.

Example 11 is the method of example 10 that may optionally include thatthe generating the probability of the lung sliding includes activatingthe neural network automatically and without user intervention based onthe one or more of the additional B-Mode ultrasound images having aquality level above a threshold quality level.

Example 12 is the method of example 10 that may optionally include thatthe B-Mode ultrasound images include a pleural line and the quality ofthe B-Mode ultrasound images is based on a location of the pleural linein the B-Mode ultrasound images.

Example 13 is the method of example 10 that may optionally include thatthe one or more of the additional B-Mode ultrasound images includesmultiple B-Mode ultrasound images; and wherein the method furthercomprises: determining redundant B-Mode ultrasound images of themultiple B-Mode ultrasound images; and excluding one or more of theredundant B-Mode ultrasound images from the generating the probabilityof the lung sliding by preventing the neural network from processingdata determined from the one or more of the redundant B-Mode ultrasoundimages.

Example 14 is the method of example 10 that may optionally includedetermining a region of interest in the one or more of the additionalB-Mode ultrasound images, wherein the generating the probability of thelung sliding is based on pixels of the one or more of the additionalB-Mode ultrasound images that are included in the region of interest andnot based on additional pixels of the one or more of the additionalB-Mode ultrasound images that are not included in the region ofinterest.

Example 15 is the method of example 14 that may optionally include thatthe determining the region of interest is based on a pleural line in theone or more of the additional B-Mode ultrasound images.

Example 16 is the method of example 10 that may optionally include thatthe instruction includes at least one of guidance to move an ultrasoundprobe, an adjustment of an imaging parameter, and a recommendation forselecting the neural network from a list of neural networks available onthe computing device.

Example 17 is an ultrasound system for determining lung sliding, wherethe ultrasound system comprises: a memory to maintain ultrasound imagesand a medical worksheet; a neural network implemented at least partiallyin hardware of the ultrasound system to generate, based on one or moreof the ultrasound images, a probability of the lung sliding; and aprocessor system to populate, automatically and without userintervention in response to the neural network generating theprobability, a field of the medical worksheet with an indicator of thelung sliding that is based on the probability.

Example 18 is the ultrasound system of example 17 that may optionallyinclude a display screen implemented to display: the medical worksheetincluding the field populated with the indicator of the lung sliding;and a request for a user confirmation of the indicator of the lungsliding.

Example 19 is the ultrasound system of example 17 that may optionallyinclude an ultrasound probe having a position sensor configured togenerate position data, wherein the processor system is implemented todetermine the field of the medical worksheet based on the position data.

Example 20 is the ultrasound system of example 17 that may optionallyinclude an ultrasound probe having a position sensor configured togenerate position data, wherein the processor system is implemented toindicate in the medical worksheet, based on the position data, lungregions scanned with the ultrasound probe.

All of the methods and tasks described herein may be performed and fullyautomated by a computer system. The computer system may, in some cases,include multiple distinct computers or computing devices (e.g., physicalservers, workstations, storage arrays, cloud computing resources, etc.)that communicate and interoperate over a network to perform thedescribed functions. Each such computing device typically includes aprocessor (or multiple processors) that executes program instructions ormodules stored in a memory or other non-transitory computer-readablestorage medium or device (e.g., solid state storage devices, diskdrives, etc.). The various functions disclosed herein may be embodied insuch program instructions, or may be implemented in application-specificcircuitry (e.g., ASICs or FPGAs) of the computer system. Where thecomputer system includes multiple computing devices, these devices may,but need not, be co-located. The results of the disclosed methods andtasks may be persistently stored by transforming physical storagedevices, such as solid-state memory chips or magnetic disks, into adifferent state. In some embodiments, the computer system may be acloud-based computing system whose processing resources are shared bymultiple distinct business entities or other users.

Depending on the embodiment, certain acts, events, or functions of anyof the processes or algorithms described herein can be performed in adifferent sequence, can be added, merged, or left out altogether (e.g.,not all described operations or events are necessary for the practice ofthe algorithm). Moreover, in certain embodiments, operations or eventscan be performed concurrently, e.g., through multi-threaded processing,interrupt processing, or multiple processors or processor cores or onother parallel architectures, rather than sequentially.

The various illustrative logical blocks, modules, routines, andalgorithm steps described in connection with the embodiments disclosedherein can be implemented as electronic hardware (e.g., ASICs or FPGAdevices), computer software that runs on computer hardware, orcombinations of both. Moreover, the various illustrative logical blocksand modules described in connection with the embodiments disclosedherein can be implemented or performed by a machine, such as a processordevice, a digital signal processor (DSP), an application specificintegrated circuit (ASIC), a field programmable gate array (FPGA) orother programmable logic device, discrete gate or transistor logic,discrete hardware components, or any combination thereof designed toperform the functions described herein. A processor device can be amicroprocessor, but in the alternative, the processor device can be acontroller, microcontroller, or state machine, combinations of the same,or the like. A processor device can include electrical circuitryconfigured to process computer-executable instructions. In anotherembodiment, a processor device includes an FPGA or other programmabledevice that performs logic operations without processingcomputer-executable instructions. A processor device can also beimplemented as a combination of computing devices, e.g., a combinationof a DSP and a microprocessor, a plurality of microprocessors, one ormore microprocessors in conjunction with a DSP core, or any other suchconfiguration. Although described herein primarily with respect todigital technology, a processor device may also include primarily analogcomponents. For example, some or all of the rendering techniquesdescribed herein may be implemented in analog circuitry or mixed analogand digital circuitry. A computing environment can include any type ofcomputer system, including, but not limited to, a computer system basedon a microprocessor, a mainframe computer, a digital signal processor, aportable computing device, a device controller, or a computationalengine within an appliance, to name a few.

The elements of a method, process, routine, or algorithm described inconnection with the embodiments disclosed herein can be embodieddirectly in hardware, in a software module executed by a processordevice, or in a combination of the two. A software module can reside inRAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory,registers, hard disk, a removable disk, a CD-ROM, or any other form of anon-transitory computer-readable storage medium. An exemplary storagemedium can be coupled to the processor device such that the processordevice can read information from, and write information to, the storagemedium. In the alternative, the storage medium can be integral to theprocessor device. The processor device and the storage medium can residein an ASIC. The ASIC can reside in a user terminal. In the alternative,the processor device and the storage medium can reside as discretecomponents in a user terminal.

Conditional language used herein, such as, among others, “can,” “could,”“might,” “may,” “e.g.,” and the like, unless specifically statedotherwise, or otherwise understood within the context as used, isgenerally intended to convey that certain embodiments include, whileother embodiments do not include, certain features, elements or steps.Thus, such conditional language is not generally intended to imply thatfeatures, elements or steps are in any way required for one or moreembodiments or that one or more embodiments necessarily include logicfor deciding, with or without other input or prompting, whether thesefeatures, elements or steps are included or are to be performed in anyparticular embodiment. The terms “comprising,” “including,” “having,”and the like are synonymous and are used inclusively, in an open-endedfashion, and do not exclude additional elements, features, acts,operations, and so forth. Also, the term “or” is used in its inclusivesense (and not in its exclusive sense) so that when used, for example,to connect a list of elements, the term “or” means one, some, or all ofthe elements in the list.

Disjunctive language such as the phrase “at least one of X, Y, or Z,”unless specifically stated otherwise, is otherwise understood with thecontext as used in general to present that an item, term, etc., may beeither X, Y, or Z, or any combination thereof (e.g., X, Y, or Z). Thus,such disjunctive language is not generally intended to, and should not,imply that certain embodiments require at least one of X, at least oneof Y, and at least one of Z to each be present.

While the above detailed description has shown, described, and pointedout novel features as applied to various embodiments, it can beunderstood that various omissions, substitutions, and changes in theform and details of the devices or algorithms illustrated can be madewithout departing from the spirit of the disclosure. As can berecognized, certain embodiments described herein can be embodied withina form that does not provide all of the features and benefits set forthherein, as some features can be used or practiced separately fromothers. The scope of certain embodiments disclosed herein is indicatedby the appended claims rather than by the foregoing description. Allchanges which come within the meaning and range of equivalency of theclaims are to be embraced within their scope.

We claim:
 1. A method implemented by a computing device for determininglung sliding, the method comprising: receiving one or more B-Modeultrasound images that include a pleural line; generating a feature listfrom the one or more B-Mode ultrasound images, the feature listindicating at least one feature of the pleural line; and generating,with a neural network implemented at least partially in hardware of thecomputing device and configured to process the feature list and a B-Modeultrasound image of the one or more B-Mode ultrasound images, aprobability of the lung sliding.
 2. The method as described in claim 1,wherein the generating the probability of the lung sliding includesactivating the neural network to process the feature list and the B-Modeultrasound image automatically and without user intervention based onthe B-Mode ultrasound image having a quality level above a thresholdquality level.
 3. The method as described in claim 1, further comprisingdetermining a region of interest in the B-Mode ultrasound image based ona location of the pleural line in the B-Mode ultrasound image; whereinthe generating the probability of the lung sliding is based on pixels ofthe B-Mode ultrasound image that are included in the region of interestand not based on additional pixels of the B-Mode ultrasound image thatare not included in the region of interest.
 4. The method as describedin claim 3, wherein the location indicates a distance from an end pointof the pleural line to an edge of the B-Mode ultrasound image, and thedetermining the region of interest is based on the distance.
 5. Themethod as described in claim 1, further comprising generating anadditional probability of the lung sliding based on an additional B-Modeultrasound image of the one or more B-Mode ultrasound images.
 6. Themethod as described in claim 5, wherein the generating the additionalprobability of the lung sliding is based on the feature list.
 7. Themethod as described in claim 5, further comprising generating anadditional feature list from the additional B-Mode ultrasound image,wherein the generating the additional probability of the lung sliding isbased on the additional feature list.
 8. The method as described inclaim 5, further comprising: merging the probability and the additionalprobability to form a merged probability of the lung sliding; anddisplaying, in a user interface of the computing device, arepresentation of the merged probability of the lung sliding.
 9. Themethod as described in claim 1, further comprising: determining anadditional B-Mode ultrasound image of the one or more B-Mode ultrasoundimages as redundant to the B-Mode ultrasound image; and discarding theadditional B-Mode ultrasound image from the one or more B-Modeultrasound images to prevent the neural network from processing theadditional B-Mode ultrasound image.
 10. A method implemented by acomputing device for determining lung sliding, the method comprising:generating B-Mode ultrasound images; determining an instruction forimproving a quality of the B-Mode ultrasound images; displaying, on auser interface of the computing device, the instruction; generatingadditional B-Mode ultrasound images based on a user adjustmentimplemented based on the instruction; and generating, with a neuralnetwork implemented at least partially in hardware of the computingdevice and based on one or more of the additional B-Mode ultrasoundimages, a probability of the lung sliding.
 11. The method as describedin claim 10, wherein the generating the probability of the lung slidingincludes activating the neural network automatically and without userintervention based on the one or more of the additional B-Modeultrasound images having a quality level above a threshold qualitylevel.
 12. The method as described in claim 10, wherein the B-Modeultrasound images include a pleural line and the quality of the B-Modeultrasound images is based on a location of the pleural line in theB-Mode ultrasound images.
 13. The method as described in claim 10,wherein the one or more of the additional B-Mode ultrasound imagesincludes multiple B-Mode ultrasound images; and further comprising:determining redundant B-Mode ultrasound images of the multiple B-Modeultrasound images; and excluding one or more of the redundant B-Modeultrasound images from the generating the probability of the lungsliding by preventing the neural network from processing data determinedfrom the one or more of the redundant B-Mode ultrasound images.
 14. Themethod as described in claim 10, further comprising determining a regionof interest in the one or more of the additional B-Mode ultrasoundimages; wherein the generating the probability of the lung sliding isbased on pixels of the one or more of the additional B-Mode ultrasoundimages that are included in the region of interest and not based onadditional pixels of the one or more of the additional B-Mode ultrasoundimages that are not included in the region of interest.
 15. The methodas described in claim 14, wherein the determining the region of interestis based on a pleural line in the one or more of the additional B-Modeultrasound images.
 16. The method as described in claim 10, wherein theinstruction includes at least one of guidance to move an ultrasoundprobe, an adjustment of an imaging parameter, and a recommendation forselecting the neural network from a list of neural networks available onthe computing device.
 17. An ultrasound system for determining lungsliding, the ultrasound system comprising: a memory to maintainultrasound images and a medical worksheet; a neural network implementedat least partially in hardware of the ultrasound system to generate,based on one or more of the ultrasound images, a probability of the lungsliding; and a processor system to populate, automatically and withoutuser intervention in response to the neural network generating theprobability, a field of the medical worksheet with an indicator of thelung sliding that is based on the probability.
 18. The ultrasound systemas described in claim 17, further comprising a display screenimplemented to display: the medical worksheet including the fieldpopulated with the indicator of the lung sliding; and a request for auser confirmation of the indicator of the lung sliding.
 19. Theultrasound system as described in claim 17, further comprising anultrasound probe having a position sensor configured to generateposition data; wherein the processor system is implemented to determinethe field of the medical worksheet based on the position data.
 20. Theultrasound system as described in claim 17, further comprising anultrasound probe having a position sensor configured to generateposition data; wherein the processor system is implemented to indicatein the medical worksheet, based on the position data, lung regionsscanned with the ultrasound probe.